Iterative Improvement of Neural Classifiers

A new objective function for neural net classifier design is presented, which has more free parameters than the classical objective function. An iterative minimization technique for the objective function is derived which requires the solution of multiple sets of numerically ill-conditioned linear equations. An enhanced feedforward network training algorithm is derived, which solves linear equations for output weights and reduces a separate error function with respect to hidden layer weights. The design method is applied to networks used to classify aerial survey imagery from remote sensing and to networks used to classify handprinted numeral image data. The improvement of the iterative technique over classical design approaches is clearly demonstrated. Introduction Two commonly used neural network classifiers are the functional link neural network (FLNN) (Pao 1989) and the multilayer perceptron (MLP) (Rumelhart & McClelland 1988). The MLP and FLNN approximate the general Bayes discriminant (Ruck et al. 1990; Wan 1990). FLNNs and MLPs are designed by minimizing the standard training error,